Assessing the Situation Awareness of Pilots Engaged in Self Spacing

Under the Next Generation Airspace Transportation System (NGATS), airspace operators will assume new roles and responsibilities in reaction to several essential changes in air traffic management operations, and the introduction of new automation technologies to support these important developments. For these changes to succeed, it is important that the information required of each new operator role be identified and the impact of these changes on operator situation awareness (SA) and workload be assessed. Because existing measures of SA and workload have not been evaluated in this environment, either existing techniques must be re-validated, or new measurement techniques must be developed that are valid, reliable and sensitive to the changes in operator SA and workload caused by NGATS. The construct of SA affects human performance in many complex systems. Intuitively, SA refers to one’s comprehension of the situation, or simply knowing what’s going on around you. SA affects human performance on tasks that have unpredictable inputs and require that responses be made either within a critical time period, or in proper sequence (i.e., before another response is made). Pilot and air traffic control (ATC) tasks are highly time-dependent, and it is not surprising that a great deal of research has been performed on SA of these operators. Intuitively, airspace operators know what is meant by SA: controllers refer to it as “having the picture;” and pilots have called it “staying ahead of the aircraft” (Endsley et al., 1998; European Air Traffic Management Programme, 2003). Despite considerable research on SA, there is no universally accepted definition or model of the concept. The most widely cited definition of SA is Endsley’s (see e.g., Endsley, 1995): SA is based on three element levels: the perception of information in the environment within a volume of time and space, the integration and understanding of the meaning of the information, and the projection of the information to future events (including knowing what further information is needed to maintain SA). In a recent Eurocontrol review, SA was defined as the continuous extraction of environmental information, the integration of this information with previous knowledge to form a coherent mental picture, and the use of that picture in directing further perception and anticipating future events (Jeannot, et al., 2003; Dominguez et al., 1994). Both definitions have been criticized, however, as not adequately reflecting the operator’s view of SA. ATCs, for example, view SA as a prioritized list of elements in the airspace environment. Ruitenberg (1997) states that terms such as “elements” or “information” do not specify exactly the nature of the elements or information. Controllers believe that SA, in addition to traffic information, includes personal factors, weather, equipage, navigational aids and performance. With these diverse definitions of SA, it is not surprising that standardized methods of SA measurement are unavailable. In the last two decades, many SA measurement methods have been advanced. In recent reviews of SA, Eurocontrol (2000) reviewed nine methods; Salmon et al. (2006) evaluated 17 methods. Nevertheless, it has been difficult to develop a measurement method that meets all of the psychometric and operational criteria of a good measurement tool (e.g., Salmon et al., 2006). Most SA measures can be classified into one of three categories, probe techniques, rating scales, or performance-correlated measures. Probe techniques administer SA-related queries during a simulation. The most commonly used probe techniques are Endsley’s “freeze-probe” technique, known as SAGAT, in which questions are asked during simulation pauses, and Durso’s Situation Present Awareness Method (SPAM) in which individual questions are asked during the course of a simulation run without scenario pauses (e.g., Endsley, 1995; Durso et al., 1997). In Endsley’s often cited studies of airtraffic-management SA (e.g., Endsley, Bolte, & Jones, 2003), SAGAT probes are typically presented during scenario freezes at random intervals. ATC is asked about operationally relevant aspects of locations and characteristics of aircraft in a sector. Pilots are typically queried about nearby aircraft. These questions are developed through Endsley’s Goal Based Task Analysis Procedure. With SPAM (Durso et al., 1997) operators are asked individual questions in the course of a scenario while performing normal tasks. According to Durso, with good SA, either task-relevant SA information is held directly in memory or the location of this information is held in memory. Therefore, in SPAM, SA is measured as both the number of correct responses and the time to answer the question correctly. If the information being queried is held in the operator’s memory, he or she can respond quickly. If the PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 52nd ANNUAL MEETING—2008 11